Electronic device with gesture-based task management

ABSTRACT

An electronic device includes a display screen for displaying ( 302 ) an active first application, a movement sensing assembly for providing signals indicative of movement of an object with respect to the display screen, and a processor in electronic communication with the movement sensing assembly and the display screen. The processor evaluates the signals from the movement sensing assembly to identify ( 304 ) a subdividing gesture, and instructs the display screen to display ( 306 ) the first application in a first portion of the display screen to one side of the subdividing gesture.

FIELD OF THE INVENTION

The present invention relates to a mobile electronic device with theability to manage and switch between applications using gestures.

BACKGROUND

Portable electronic devices such as smart phones, personal digitalassistants (PDAs), and tablets have become popular and ubiquitous. Moreand more features have been added to these devices, and they are oftenequipped with powerful processors, significant memory, and openoperating systems, which allows for many different applications to beadded. Popular applications provide functions such as calling, emailing,texting, image acquisition, image display, music and video playback,location determination (e.g., GPS), and internet browsing functions,among others. Further, such devices often include various user inputcomponents for communicating instructions to control operation of theelectronic device. For example, many mobile devices are equipped notonly with various buttons and/or keypads, but also with touch detectingsurfaces (such as touch screens or touch pads) by which a user, simplyby touching a particular area of the mobile device and/or by moving afinger along the surface of the mobile device, is able to communicateinstructions to control the electronic device.

Often mobile electronic devices (such as smart phones) have limiteddisplay screen and user interface surface area due to the desire to keepthe device portable. Generally with such devices, only a singleapplication can be viewed at one time. Further, switching betweenapplications often requires closing or at least pausing a currentlyrunning application. Thus, it is desirable to provide an electronicdevice with features to address these concerns.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a front perspective view of an exemplary electronic device;

FIG. 2 is a block diagram of the exemplary electronic device of FIG. 1;

FIG. 3 illustrates an exemplary method for the electronic device of FIG.1; and

FIGS. 4-11 are front views of the exemplary electronic device of FIG. 1illustrating the method of FIG. 3.

DETAILED DESCRIPTION

A mobile electronic device with a display screen and a touch detectingsurface (forming a touch screen) generates signals representative ofuser gestures. When an active application is running and displayed onthe display screen, a user can divide the display screen into twoportions with a subdividing gesture, which is detected and identifiedwith the touch detecting surface and a processor. When a validsubdividing gesture is identified, the location of the subdividinggesture indicates a boundary between the two portions. The device isthen controlled such that the active application is reduced in size anddisplayed in one of the portions. Optionally, a set of application iconsfor selection is displayed in the other portion. The user can thenselect a desired one of the application icons, and the applicationassociated with the selection can be launched and displayed in the otherportion (replacing the application icons). In this manner, two or moreapplications can be displayed at the same time, and data can betransferred between them using commonly known functions such as cut andpaste, drag and drop, and the like. Further, the first activeapplication can then be closed or minimized, and the second activeapplication can be increased in size to be displayed on the entiredisplay screen. In such a manner, simple and intuitive gestures can beused to allow for multiple applications to be displayed and managed, andto switch between applications.

Referring now to FIG. 1, an exemplary mobile electronic device 102 isillustrated which can take the form of a mobile phone (as more fullydescribed with respect to FIG. 2) and can include functions such ascalling, emailing, texting, image acquisition, and internet browsingfunctions, as well as others. In other embodiments, the mobileelectronic device can be one of a variety of other devices such as apersonal digital assistant, remote controller, electronic book reader,or tablet. The electronic device 102 includes a movement sensingassembly, which in FIG. 1 takes the form of a touch detecting surface104 associated with a display screen 106 to form a touch screen. Thetouch detecting surface 104 can be any of a variety of known touchdetecting technologies such as a resistive technology, a capacitivetechnology, or an optical technology. As illustrated, the touchdetecting surface 104 includes a light permeable panel or othertechnology which overlaps display screen 106 (such as a liquid crystaldisplay screen). Alternately, the movement sensing assembly could be atouchpad (not overlapping the display screen), a joystick, a mouse, orother types of graphic user interfaces.

The movement sensing assembly can alternately take other forms such asthe sensing assembly shown and described in U.S. patent application Ser.No. 12/471,062, titled “Sensing Assembly For Mobile Device” and filed onMay 22, 2009. For example, such a sensing assembly can include aplurality of phototransmitters arranged to emit light outwardly invarious directions, with at least one photoreceiver arranged to receiverespective portions of transmitted light originating from eachphototransmitter that has been reflected off an object (otherconfigurations of phototransmitters and photoreceivers are alsopossible), and can also detect and identify various user gestures.

Optionally, the electronic device 102 may include a keypad 108 havingnumerous keys for inputting various user commands for operation of thedevice. Although the keypad 108 is shown as a QWERTY keypad, the keypadmay be implemented as a Bell Telephone keypad (1-9, *, 0, #), a customkeypad with buttons such as ‘home’ and ‘backspace’, or another type ofkeypad with programmable “soft” keys.

The electronic device 102 is operable to detect and identify variousgestures by a user (where each gesture is a specified pattern ofmovement of an external object, such as a hand or one or more fingers,relative to the device), in one of a variety of known ways. A touchscreen is advantageous because changeable graphics can be displayeddirectly underlying the touch detecting surface on which controllinghand gestures are applied.

Referring to FIG. 2, a block diagram 200 illustrates exemplary internalcomponents of the mobile smart phone implementation of the electronicdevice 102. These components can include wireless transceivers 202, aprocessor 204 (e.g., a microprocessor, microcomputer,application-specific integrated circuit, or the like), memory 206, oneor more output components 208, one or more input components 210, and oneor more sensors 228. The device can also include a component interface212 to provide a direct connection to auxiliary components oraccessories for additional or enhanced functionality, and a power supply214, such as a battery, for providing power to the other internalcomponents. All of the internal components can be coupled to oneanother, and in communication with one another, by way of one or moreinternal communication links 232, such as an internal bus.

The memory 206 can encompass one or more memory devices of any of avariety of forms (e.g., read-only memory, random access memory, staticrandom access memory, dynamic random access memory, etc.), and can beused by the processor 204 to store and retrieve data. The data that isstored by the memory 206 can include operating systems, applications,and informational data. Each operating system includes executable codethat controls basic functions of the electronic device, such asinteraction among the various internal components, communication withexternal devices via the wireless transceivers 202 and/or the componentinterface 212, and storage and retrieval of applications and data to andfrom the memory 206.

As for programs (applications), each program includes executable codethat utilizes an operating system to provide more specificfunctionality, such as file system service and handling of protected andunprotected data stored in the memory 206. Although many such programsgovern standard or required functionality of the electronic device 102,in many cases the programs include applications governing optional orspecialized functionality, which can be provided in some cases by thirdparty vendors unrelated to the electronic device manufacturer.

Finally, with respect to informational data, this is non-executable codeor information that can be referenced and/or manipulated by an operatingsystem or program for performing functions of the mobile device 102.Such informational data can include, for example, data that ispreprogrammed upon the electronic device 102 during manufacture, or anyof a variety of types of information that is uploaded to, downloadedfrom, or otherwise accessed at servers or other devices with which theelectronic device 102 is in communication during its ongoing operation.

The mobile device 102 can be programmed such that the processor 204 andmemory 206 interact with the other components of the electronic deviceto perform a variety of functions, including interaction with the touchdetecting surface 104 to receive signals indicative of gesturestherefrom, evaluation of these signals to identify various gestures, andcontrol of the device in the manners described below.

Although not specifically shown in FIG. 2, the processor can includevarious modules for detecting different gestures, such as single-fingerglide (including its location), two-finger glide (including itslocation), tap (including its location), double-tap (including itslocation), and hold-and-drag (including its location) gesture detectionmodules. Further the processor can include various modules forinitiating different activities such as launching an application,subdividing the display screen into portions, displaying a set ofapplication icons in a portion, resizing an application from full-screensize such that it fits within a portion of the display screen, and viceversa (minimize and maximize functions), and data transfer functions.Detection of certain gestures (including the gesture's location andtiming) may trigger specific activity modules. Thus, when asingle-finger glide gesture detection module concludes that asingle-finger has been swiped from one edge of the display screen to anopposite edge, it may trigger the subdividing activity module.

The wireless transceivers 202 can include both cellular transceivers 203and a wireless local area network (WLAN) transceiver 205. Each of thewireless transceivers 202 utilizes a wireless technology forcommunication, such as cellular-based communication technologiesincluding analog communications (using AMPS), digital communications(using CDMA, TDMA, GSM, iDEN, GPRS, EDGE, etc.), and next generationcommunications (using UMTS, WCDMA, LTE, IEEE 802.16, etc.) or variantsthereof, or peer-to-peer or ad hoc communication technologies such asHomeRF, Bluetooth and IEEE 802.11 (a, b, g or n), or other wirelesscommunication technologies.

Exemplary operation of the wireless transceivers 202 in conjunction withothers of the internal components of the electronic device 102 can takea variety of forms and can include, for example, operation in which,upon reception of wireless signals, the internal components detectcommunication signals and the transceiver 202 demodulates thecommunication signals to recover incoming information, such as voiceand/or data, transmitted by the wireless signals. After receiving theincoming information from the transceiver 202, the processor 204 formatsthe incoming information for the one or more output components 208.Likewise, for transmission of wireless signals, the processor 204formats outgoing information, which may or may not be activated by theinput components 210, and conveys the outgoing information to one ormore of the wireless transceivers 202 for modulation as communicationsignals. The wireless transceiver(s) 202 convey the modulated signals toa remote device, such as a cell tower or an access point (not shown).

The output components 208 can include a variety of visual, audio, and/ormechanical outputs. For example, the output components 208 can includeone or more visual output components 216 such as the display screen 106.One or more audio output components 218 can include a speaker, alarm,and/or buzzer, and one or more mechanical output components 220 caninclude a vibrating mechanism for example. Similarly, the inputcomponents 210 can include one or more visual input components 222 suchas an optical sensor of a camera, one or more audio input components 224such as a microphone, and one or more mechanical input components 226such as the touch detecting surface 104 and the keypad 108 of FIG. 1.

The sensors 228 can include both proximity sensors 229 and other sensors231, such as an accelerometer, a gyroscope, or any other sensor that canprovide pertinent information, such as to identify a current location ororientation of the device 102.

Actions that can actuate one or more input components 210 can includefor example, powering on, opening, unlocking, moving, and/or operatingthe device 102. For example, upon power on, a ‘home screen’ with apredetermined set of application icons can be displayed on the displayscreen 106.

FIG. 3 illustrates a method 300 that can be performed by the electronicdevice 102, such as at a time when a set of application icons forselection are displayed on the display screen. The method begins at astep 302, wherein a desired first application is launched and displayedon the display screen in response to a user input command, such as atouch gesture selecting a desired one of the displayed applicationicons. An example of a first running application 400 displayed ondisplay screen 106 is illustrated in FIG. 4. Due to the small screensize of portable electronic devices, active applications are usuallydisplayed in a full-screen format as shown. Processing then proceeds toa step 304.

At step 304, the device 102 monitors signals from the touch detectingsurface to detect a valid gesture for subdividing the display into afirst portion and a second portion. For example, as schematicallyillustrated in FIG. 5, a valid subdividing gesture can be a slidegesture (indicated by arrow 500), which is movement of an object such asa finger generally in the xy plane in one direction with respect to thedevice. The illustrated slide gesture extends from the top to the bottomof the display screen 106 to define a left side portion 502 and a rightside portion 504 of the display screen. A slide gesture extending fromone side to the other side can also be a valid subdividing gesture, andcan operate to subdivide the display screen into a top portion and abottom portion. Another possible subdividing gesture can be a gesturethat defines a bounded area 900 within the display screen, asschematically illustrated in FIG. 9. Processing then proceeds to a step306.

Note that the slide gesture 500 demarcated the boundary 511 between theleft side portion 502 and the right side portion 504. Although a linear,orthogonal gesture 500 is shown, due to human factors, the gestureitself may not be perfectly linear or orthogonal and the method 300 mayinterpret the gesture to be linear and orthogonal when creating theboundary 511. Additionally, the method may accept linear, slantedgestures, curved gestures, three-dimensional, and other types ofgestures. Also, although touch screen gestures are described here (wherea user directly touches an image to provide input), other gestures maybe used, such as mouse or joystick gestures (where a pointing device isused to manipulate a pointer icon on an image) or three-dimensionalgestures in space.

At step 306, the device 102 operates to display the first runningapplication 400 in the first portion 502 (at a reduced size). At step307, the device operates to display a predetermined set of applicationicons 602 in the second portion 504, such as illustrated in FIG. 6. Thepredetermined set of application icons 602 can include, for example,icons which correspond to one or more installed applications, recentlyclosed applications, favorite applications, and applications running inthe background. As shown, the application 400 is displayed on the leftside portion 502, and the application icons are displayed on the rightside portion 504, although in other embodiments, which portion receivesthe reduced size application can be set as a default, as a preference,or can be changed using a gesture. For example, a default oruser-programmable setting may consistently place the first runningapplication in the smaller portion. Alternately, a default oruser-programmable setting may consistently place the first runningapplication in a left-side or bottom-side portion. As an additionalexample, a two-finger slide gesture from left to right can operate todrag the contents of portion 502 to portion 504, while the contents ofportion 504 are transferred to portion 502. Processing then proceeds toa step 308.

At step 308, while the application icons are displayed, the devicemonitors signals from the touch detecting surface 104 to detect a validgesture for selecting one of the application icons. Such a selectiongesture can be a simple touch by a user's finger 604 (or stylus) in anarea corresponding to a desired selected icon, as schematicallyillustrated in FIG. 6. Processing then proceeds to a step 310.

At step 310, the device 102 operates to launch and display a secondapplication, which is the application corresponding to the selectedicon, in the second portion 504 of the display screen. In this manner,the first application 400 is displayed in the first portion 502 and thesecond application 700 is displayed in the second portion of the displayscreen, such as illustrated in FIG. 7. Processing then proceeds to astep 312.

At step 312, while both applications are displayed, the electronicdevice monitors for data transfer commands, and in response operates toeffect data transfer between the first and the second applications,using known techniques such as cut and paste or drag and drop.Processing then proceeds to a step 314.

At step 314, while both running applications are displayed, the devicemonitors signals from the touch detecting surface to detect a validgesture for closing one of the running applications, or a valid gesturefor maximizing one of the running applications. For example, a closegesture can be accomplished by a pinching in of two fingers in theportion of the display screen which includes the application to beclosed, or perhaps a single touch in an appropriately marked ordesignated ‘close’ location corresponding to one of the runningapplications. Similarly, a maximize gesture can be accomplished bymoving two fingers apart in either the first or the second portion, orperhaps a single touch in an appropriately marked or designated maximizelocation corresponding to one of the running applications. Processingthen proceeds to a step 316.

At step 316, the device 102 acts to resize the application correspondingto the maximize gesture, while closing the other application (or havingit run in the background), such as illustrated in FIG. 8, wherein thepresumed selected application (the running second application 700) isdisplayed on substantially the entire display screen. Alternately, thedevice acts to close the application corresponding to the close gesture,while increasing the size of the other application.

At a step 318, it is determined whether to continue with the currentapplication 700 as the launched application. If yes, the processing canagain to proceed to step 304. If no, for example, if an exit command isdetected, then the process ends at 320.

FIGS. 9-11 provide another example of the operation of the electronicdevice. FIG. 9 shows the display screen divided into a first portion 900and a second portion 902. FIG. 10 shows a set of application icons 1000which can then displayed in portion 900. FIG. 10 illustrates thatanother application 1100 can be selected to be displayed in portion 900while the application 700 is displayed in portion 902.

In this manner, gestures provide a user friendly and intuitive way tomanage and switch between tasks. Although the implementations describedher show a single subdividing gesture for dividing a display into twoportions (e.g., a line “|” or an enclosed area “□”), a doublesubdividing gesture can be used to divide the display into four portions(e.g., a cross or plus “+”), or three portions (e.g. a “T”), and othergestures or series of gesture can also be defined as valid subdividinggestures.

As understood by those in the art, processor 204 executes computerprogram code to implement the methods described herein. Embodimentsinclude computer program code containing instructions embodied intangible media, such as floppy diskettes, CD-ROMs, hard drives, or anyother computer-readable storage medium, wherein, when the computerprogram code is loaded into and executed by a processor, the processorbecomes an apparatus for practicing the invention. Embodiments includecomputer program code, for example, whether stored in a storage medium,loaded into and/or executed by a computer, or transmitted over sometransmission medium, such as over electrical wiring or cabling, throughfiber optics, or via electromagnetic radiation, wherein, when thecomputer program code is loaded into and executed by a computer, thecomputer becomes an apparatus for practicing the invention. Whenimplemented on a general-purpose microprocessor, the computer programcode segments configure the microprocessor to create specific logiccircuits.

It is specifically intended that the present invention not be limited tothe embodiments and illustrations contained herein, but include modifiedforms of those embodiments, including portions of the embodiments andcombinations of elements of different embodiments as come within thescope of the following claims.

The invention claimed is:
 1. An electronic device comprising: a displayscreen; a movement sensing assembly configured to provide one or moresignals indicative of a movement of an object with respect to thedisplay screen; and a processor in electronic communication with themovement sensing assembly and the display screen, wherein the processoris configured to: execute a first application; evaluate the one or moresignals to identify a subdividing gesture that defines at least oneboundary between first and second portions of the display screen;output, for display in the first portion of the display screen,information associated with the first application; output, for displayin the second portion of the display screen, a plurality of applicationicons; evaluate the one or more signals to identify a selection gesturethat selects one of the plurality of application icons; execute a secondapplication that corresponds to the selected one of the plurality ofapplication icons; and output, for display at the display screen,information associated with the second application.
 2. The electronicdevice of claim 1, wherein the subdividing gesture circumscribes anenclosed area of the display screen, wherein the first portion of thedisplay screen is outside of the enclosed area, wherein the secondportion of the display screen is inside of the enclosed area, andwherein the enclosed area is a quadrilateral.
 3. The electronic deviceof claim 1, wherein, prior to evaluating the one or more signals toidentify the subdividing gesture, the processor is configured to output,for display at the display screen, the information associated with thefirst application in a full screen format.
 4. The electronic device ofclaim 1, wherein the display screen and the movement sensing assemblyare incorporated into a touch screen.
 5. The electronic device of claim1, wherein the movement sensing assembly is a touch detecting surface.6. The electronic device of claim 1, wherein the movement sensingassembly includes at least one phototransmitter and at least onephotoreceiver.
 7. The electronic device of claim 1, wherein theplurality of application icons correspond to at least one of: a group ofinstalled applications, a group of recently used applications, or agroup of applications running in background.
 8. The electronic device ofclaim 1, wherein the processor is further configured to: evaluate theone or more signals to identify a data transfer gesture; and responsiveto identifying the data transfer gesture, transfer data from the firstapplication to the second application.
 9. The electronic device of claim1, wherein the processor is further configured to: after outputting theinformation associated with the first application in the first portionof the display screen, evaluate the one or more signals to identify aclose gesture corresponding to the first application; and responsive toidentifying the close gesture, output, for display at the displayscreen, the information associated with the second application in afull-screen format.
 10. The electronic device of claim 1, wherein theprocessor is further configured to: after outputting the informationassociated with the first application in the first portion of thedisplay screen, evaluate the one or more signals to identify a maximizegesture corresponding to the first application; and responsive toidentifying the maximize gesture, output, for display at the displayscreen, the information associated with the first application in afull-screen format.
 11. The electronic device of claim 1, wherein theprocessor is configured to output, for display in the second portion ofthe display screen, the information associated with the secondapplication.
 12. A method comprising: executing, by an electronicdevice, a first application; outputting, by the electronic device andfor display in a full-screen format at a display screen, informationassociated with the first application; identifying, by the electronicdevice, a subdividing gesture that defines at least one boundary betweenfirst and second portions of the display screen; outputting, by theelectronic device and for display in the first portion of the displayscreen, the information associated with the first application;outputting, by the electronic device and for display in the secondportion of the display screen, a plurality of application icons;identifying, by the electronic device, a selection gesture that selectsone of the plurality of application icons; executing, by the electronicdevice, a second application that corresponds to the selected one of theplurality of application icons; and outputting, by the electronic deviceand for display at the display screen, information associated with thesecond application.
 13. The method of claim 12, further comprising:after outputting the information associated with the first applicationin the first portion of the display screen, identifying, by theelectronic device, a close gesture corresponding the first application;and responsive to identifying the close gesture, outputting, for displayat the display screen, the information associated with the secondapplication in a full-screen format.
 14. The method of claim 12, furtherincluding: identifying, by the electronic device, a data transfergesture; and transferring, by the electronic device, data from the firstapplication to the second application.
 15. The method of claim 12,wherein the subdividing gesture circumscribes an enclosed area of thedisplay screen, wherein the first portion of the display screen isoutside of the enclosed area, wherein the second portion of the displayscreen is inside of the enclosed area, and wherein the enclosed area isa quadrilateral.
 16. The method of claim 12, wherein the plurality ofapplication icons correspond to at least one of: a group of installedapplications, a group of recently used applications, or a group ofapplications running in background.
 17. The method of claim 12, furthercomprising: after outputting the information associated with the firstapplication in the first portion of the display screen, identifying, bythe electronic device, a maximize gesture corresponding to the firstapplication; and responsive to identifying the maximize gesture,outputting, for display at the display screen, the informationassociated with the first application in a full-screen format.
 18. Themethod of claim 12, wherein outputting the information associated withthe second application comprises outputting, by the electronic deviceand for display in the second portion of the display screen, theinformation associated with the second application.